Abstract: For space robotic manipulation, multi view cameras are generally used to provide visual information guidance for the robotic arm. In such a configuration, the global camera guides the long distance operation while the local camera guides the proximity operation. To increase the autonomy level of multi view visual information utilization, a novel method called multi view visual goal generation is proposed in this paper, which adopts a hierarchical architecture. The goal generator maps the multi view input images to visual goals expressed in SE(3) space.Furthermore, a multi view goal decision mechanism is established according to the relative error of the generated visual goal. Experimental results demonstrate that the proposed method can respond quickly (>30 fps) to numerous manipulation objects at different scales, and meanwhile show the robustness to static and dynamic image occlusion. The capability on long sequence complex manipulation tasks with visuo tactile integration is also verified.

Key words: space robotic manipulation, vision based robotic manipulation, visual goal generation, visuo tactile integration, multi task learning